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The  Army  has  spent  over  $765  million  of  the  $1  billion  estimated  total  cost 
for  the  Maneuver  Control  System  (mcs)  which  is  to  provide  battlefield 
information  to  maneuver  commanders.  Since  1980,  the  mcs  program  has 
experienced  numerous  problems,  such  as  fielding  inadequate  computer 
software  and  canceling  the  development  of  one  software  version  due  to 
design  flaws,  cost  growth,  and  schedule  slips.  Given  the  program’s  past 
difficulties  and  the  important  role  of  MCS  in  the  Army’s  battlefield 
automation  efforts,  we  reviewed  the  Army’s  development  and  acquisition 
plans  for  mcs.  Specifically,  our  objectives  were  to  determine  whether 
(1)  the  current  mcs  software  development  strategy  is  appropriate  to 
overcome  prior  development  problems  and  (2)  207  new  computers  for  mcs 
related  training  should  be  procured  as  planned. 


Background 


The  goal  of  the  Army’s  mcs  program  is  to  develop  and  field  a  computer 
system  that  provides  automated  critical  battlefield  assistance  to  maneuver 
commanders  and  their  battle  staff  at  the  corps-to-battalion  level,  mcs  is 
intended  to  enable  the  command  staff  to  collect,  store,  process,  display, 
and  disseminate  critical  data  to  produce  and  communicate  battle  plans, 
orders,  and  enemy  and  friendly  situational  reports.  It  is  a  key  component 
of  the  Army  Tactical  Command  and  Control  System,  which  is  also 
intended  to  enhance  the  coordination  and  control  of  combat  forces 
through  automated  management  of  five  key  battlefield  areas,  including 
maneuver  control.*  Given  its  role  to  communicate  battle  plans,  orders,  and 
enemy  and  friendly  situation  reports,  mcs  is  also  a  key  component  of  the 
Army’s  ongoing  efforts  to  digitize  (automate)  its  battlefield  operations. 

In  1980,  the  Army  fielded  the  first  mcs  system — ^with  limited  command, 
control,  and  communications  capabilities — to  VII  Corps  in  Europe.  In 
1982,  the  Army  awarded  a  5-year  contract  to  continue  mcs  development, 
and  by  1986  mcs  software  had  evolved  to  version  9,  also  fielded  in  Europe. 
In  1987,  the  Army  performed  post-deployment  tests  on  version  9  in 
Germany.  The  results  of  those  tests  led  the  Army  Materiel  Systems 


'The  other  battlefield  functional  areas  are  air  defense,  fire  support,  intelligence  and  electronic  warfare, 
and  combat  service  support,. 
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Analysis  Activity  to  conclude  that  mcs  did  not  exhibit  adequate  readiness 
for  field  use  and  recommend  that  further  fielding  not  occur  until  the 
system’s  problems  were  resolved.^  However,  the  Army  awarded  a  second 
5-year  contract  that  resulted  in  version  10,  which  was  fielded  by  April  1989 
and  remains  in  the  field  today.  In  November  1989,  the  Army  Materiel 
Systems  Analysis  Activity  reported  that  mcs  had  met  only  30  percent  of  its 
required  operational  capabilities  and  again  recommended  that  the  system 
not  be  released  for  field  use.  In  May  1990,  operational  testers  again 
questioned  the  system’s  functional  ability  and  effectiveness  because  it 
could  not  produce  timely,  accurate,  and  useful  information  in  a  battle 
environment. 

While  earlier  versions  of  MCS  were  being  fielded  and  withdrawn,  the 
development  of  software  continued.  In  1988,  the  Army  awarded  a  contract 
for  the  development  of  version  11.  By  February  1993,  the  Army  stopped 
development  of  version  11  software  due  to  multiple  program  slips,  serious 
design  flaws,  and  cost  growth  concerns.  The  program  was  then 
reorganized  with  a  plan  approved  by  the  Office  of  the  Secretary  of  Defense 
in  April  1993.  Under  the  reorganized  program,  a  group  of  contractors  and 
government  software  experts  have  been  working  to  develop  the  next 
version  of  mcs  software — ^version  12.01 — utilizing  software  segments  that 
could  be  salvaged  from  the  failed  version  11  effort. 

In  addition  to  software,  the  mcs  system  consists  of  computers  procured 
under  the  Army’s  Common  Hardware  and  Software  (CHS)  effort,  which  was 
undertaken  to  reverse  the  proliferation  of  program-unique  computers  and 
software.  The  Army  planned  to  acquire  288  of  the  CHS  computers  in  fiscal 
years  1997  and  1998  to  support  the  mcs  training  base,  and  has  already 
acquired  81.  Those  computers  were  used  in  a  training  base  assessment  to 
support  a  decision  to  acquire  the  remaining  207  computers. 


Results  in  Brief 


Since  its  1993  reorganization,  the  Maneuver  Control  System  has  continued 
to  experience  development  problems.  The  initial  operational  test  and 
evaluation  of  version  12.01  software  has  slipped  28  months,  from 
November  1995  to  March  1998,  and  interim  tests  have  shown  that 
significant  software  problems  continue.  Despite  these  problems,  the  Army 
awarded  a  contract  in  September  1996  for  the  concurrent  development  of 
the  next  software  versions — 12.1,  12.2,  and  12.3 — which  are  being 
developed  by  a  new  contractor  and  may  involve  substantially  different 


-In  April  1984,  the  Army  Materiel  Command  designated  the  Army  Materiel  Systems  Analysis  Activity  as 
its  independent  evaluator  for  materiel  releases  of  major  and  other  high-visibility  systems. 
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software.  If  the  Army’s  current  development  strategy  for  the  Maneuver 
Control  System  is  not  strengthened,  development  problems  may  continue 
to  occur.  Currently,  the  Army’s  strategy  allows  (1)  less  than  full 
operational  testing  of  version  12.1  and  (2)  development  of  follow-on 
versions  12.2  and  12.3  to  start  about  18  months  before  the  operational 
testing  of  each  version’s  predecessor. 

Despite  the  fact  that  the  Maneuver  Control  System  has  yet  to  undergo  an 
initial  operational  test  and  evaluation  or  be  approved  for  production,  the 
Army  plans  to  acquire  207  computers  in  fiscal  years  1997  and  1998  to 
increase  the  number  computers  available  for  system  training.  Program 
officials  stated  that  they  need  to  acquire  the  computers  before  operational 
testing  to  provide  not  only  mcs  specific  training  but  also  training  for  the 
larger  Army  Battle  Command  System,  of  which  the  Army  Tactical 
Command  and  Control  System  and  the  Maneuver  Control  System  are 
major  components.  The  207  computers,  however,  are  not  needed  to  satisfy 
any  of  the  three  legislated  reasons  for  low-rate  initial  production  before  an 
initial  operational  test  and  evaluation.® 


Development 
Problems  Continue 


Since  its  reorganization  in  1993,  mcs  program  experience  indicates 
continuing  problems  in  the  system’s  development.  Specifically,  (1)  the  mcs 
initial  operational  test  and  evaluation  of  version  12.01  has  slipped  twice, 
(2)  interim  developmental  level  tests  and  a  customer  test  done  to  support 
a  decision  to  award  a  contract  to  develop  follow-on  software  show  that 
significant  problems  continue,  and  (3)  development  of  follow-on  version 
12.1  was  begun  despite  the  results  of  the  customer  test  and  prior  program 
history. 


Opsrational  Testing  After  the  1993  program  reorganization,  version  12.01  was  scheduled  to 

Schedules  Slip  undergo  initial  operational  testing  and  evaluation  in  November  1995.  The 

test  slipped  to  November  1996  and  is  now  scheduled  for  March  1998. 
Program  officials  stated  that  the  test  date  slipped  initially  because  the  CHS 
computers  to  be  used  were  not  yet  available. 

During  August  and  September  1996,  version  12.01  underwent  a  system 
confidence  demonstration  to  determine  whether  it  was  ready  for  the 


Title  10  U.S.C.  2400  provides  that  low-rate  initial  production  of  systems,  except  for  ships  and 
satellites,  is  to  produce  the  minimum  quantity  necessary  to  (1)  provide  production-configured  or 
representative  articles  for  operational  test  and  evaluation,  (2)  establish  an  initial  production  base  for 
the  system,  and  (3)  permit  an  orderly  increase  in  the  production  rate  for  the  system  sufficient  to  lead 
to  full-rate  production  upon  the  successful  completion  of  operational  test  and  evaluation. 


Page  3 


GAO/NSIAD-98-15  Battlefield  Automation 


B-276830 


November  1996  initial  operational  test  and  evaluation.  Because  the 
software  was  not  ready,  further  work  and  two  additional  system 
confidence  demonstrations  followed  in  August  and  September  1996.  Both 
demonstrations  indicated  that  the  system  was  not  ready  for  operational 
testing.  Additionally,  the  software  still  had  an  open  priority  one  software 
deficiency  and  priority  three  and  four  deficiencies  that  would  have 
negatively  impacted  the  conduct  of  the  operational  test.^ 

Both  the  Army’s  Operational  Test  and  Evaluation  Command  and  the 
Department  of  Defense’s  (dod)  Director  of  Operational  Test  and 
Evaluation  (dot&e)  had  stated  that  there  could  be  no  open  priority  one  or 
two  software  deficiencies  before  the  operational  test.  They  had  also  stated 
that  there  could  not  be  any  open  priority  three  and  four  deficiencies  that, 
in  combination,  were  likely  to  have  a  detrimental  effect  on  the  system’s 
performance,  dot&e  staff  told  us  that  there  were  a  number  of  open  priority 
three  and  four  software  deficiencies  that  they  believe  would  have  had  a 
detrimental  effect.  When  MCS  program  officials  realized  that  these 
deficiencies  would  not  be  resolved  in  time  for  the  initial  operational  test, 
they  downgraded  the  test  3  weeks  before  it  was  to  occur  to  a  limited  user 
test,®  utilizing  $8.5  million  appropriated  for  the  mcs  operational  test  in 
fiscal  years  1996  and  1997.®  That  test  was  conducted  in  November  1996. 
While  the  test  report  has  not  been  finalized,  a  draft  version  states  that 
MCS — in  the  tested  configuration — is  not  operationally  effective  or  suitable. 


Interim  Development  Level 
Tests  Indicate  Continuing 
Problems 


Throughout  the  development  of  version  12.01,  interim  software  builds 
have  undergone  numerous  performance  tests  to  determine  the  current 
state  of  software  development,  and  build  4  was  subjected  to  a  customer 


'‘Software  deficiencies  are  rated  in  a  priority  system,  from  priority  one — the  most  critical — to  priority 
five — the  least  critical.  An  open  software  deficiency  is  a  deficiency  identified  through  testing  that  is 
not  considered  to  be  resolved. 

•'^e  limited  user  test  involved  the  same  testing  planned  for  the  initial  operational  test  and  evaluation. 

It  was  limited  in  that  it  had  no  pass/fail  criteria  and  was  not  to  be  used  to  support  a  full-rate  production 
decision.  It  served  as  a  learning  experience,  providing  information  on  the  current  maturity  of  the  MCS 
software  and  a  baseline  of  performance  by  which  to  judge  future  development  efforts. 

'’MCS  program  officials  stated  that  at  the  time  it  became  apparent  that  MCS  was  not  ready  for  its  initial 
operational  test  and  evaluation,  it  made  sense  to  go  forward  with  the  test  as  a  limited  user  test  because 
the  user  had  been  trained;  the  equipment  was  instrumented  for  the  test;  and  the  users,  equipment,  and 
testers  were  all  in  place.  In  pro’viding  technical  comments  on  a  draft  of  this  report,  DOD  stated  that, 
given  the  sunk  costs,  the  Army’s  decision  to  go  forward  with  the  test  made  sense  because  an 
operational  test  could  pro’vide  invaluable  feedback  to  the  MCS  developers  that  could  not  be  obtained 
through  technical  testing. 
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Concurrent  Contract  Was 
Awarded  for  Follow-on 
Software  Development 


test/  The  results  of  those  tests  identified  continuing  problems  as  the 
number  of  builds  proceeded.  For  example,  a  December  1995  performance 
test  report  on  build  3.0  stated  that,  if  the  problems  found  during  the  test 
were  not  quickly  corrected  in  build  3.1,  then  the  risk  to  the  program  might 
be  unmanageable.  The  follow-on  April  1996  performance  test  report  of 
build  3.1  stated  that  significant  problems  in  system  stability  prevented 
proper  testing  of  several  requirements.  The  report  further  stated  that 
messaging  between  battlefield  functional  areas  was  extremely  difficult  and 
problematic  and  that  the  system  had  other  stability  problems. 

A  September  1996  performance  test  report  stated  that  of  568  previously 
open  deficiency  reports  from  builds  5.1  through  5.2c,  165,  almost 
29  percent,  still  remained  open.  This  report,  the  last  published  on  an  Mcs 
performance  test,  reflected  the  state  of  the  mcs  software  shortly  before  the 
downgraded  limited  user  test,  in  which  mcs  failed  to  demonstrate  either 
operational  effectiveness  or  suitability.  More  recent  performance  tests  of 
later  builds  have  been  done;  however,  separate  reports  on  those  test 
events  have  not  been  issued.  Rather,  the  program  office  plans  to  prepare 
an  integrated  test  report  in  October  or  November  1997. 


In  April  1994,  the  mcs  program  office  released  a  plan  to  begin  follow-on 
software  development  while  version  12.01  was  still  in  development.  In  a 
May  1995  memorandum,  the  Deputy  dot&e  expressed  concern  regarding 
this  plan.  He  stated  that,  because  version  12.01  was  being 

“developed  by  a  confederation  of  contractors  who  have  built  this  current  version  of  mcs  on 
the  salvaged  ’good’  portions  of  the  abruptly  terminated  development  of  mcs  Version  11,  it 
needs  to  stand  the  rigor  of  an  Independent  Operational  Test  and  Evaluation . . .  before  a 
MCS  Block  IV  [post  version  12.01  development]  contract  is  awarded.” 

To  help  determine  the  level  of  risk  in  proceeding  under  the  Army’s 
development  strategy,  dot&e  stated  in  a  June  1995  memorandum  that  an 
operational  test  of  version  12.01  be  conducted  to  measure  the  software’s 
maturity  before  the  award  of  a  contract  for  the  development  of  follow-on 
versions.  As  a  result,  an  operational  assessment — called  the  mcs  customer 
test — ^was  conducted  on  version  12.01  in  April  1996  to  support  the  award 
of  a  $63. 1  million  contract  for  the  development  of  mcs  Block  IV 
software — mcs  versions  12.1,  12.2,  and  12.3. 


software  build  involves  additions  or  changes  to  the  software  to  add  new  functions  or  correct 
deficiencies  in  the  prior  build  software.  Development  of  a  new  software  version  under  the  evolutionary 
software  development  philosophy  is  accomplished  by  multiple  intraversion  software  builds. 
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No  pass/fail  criteria  were  set  for  the  customer  test.  However,  dot&e 
directed  that  four  operational  issues  be  tested.  Those  issues  related  to 
(1)  the  capacity  of  the  system  to  store  and  process  required  types  and 
amounts  of  data,  including  the  ability  of  the  staff  users  to  frequently 
update  the  information  database;  (2)  the  capabilities  of  the  mcs  network  to 
process  and  distribute  cmrent  and  accurate  data  using  the  existing 
communications  systems;  (3)  the  impact  of  computer  server  outages  on 
continuity  of  operations;  and  (4)  the  system  administration  and  control 
capabilities  to  initialize  the  system,  become  fully  operational,  and  sustain 
operations. 

In  its  report  on  the  customer  test,  the  Army’s  Test  and  Experimentation 
Command  stated  that,  at  the  time  of  the  test,  mcs  was  evolving  from  a 
prototype  system  to  one  ready  for  initial  operational  test  and  evaluation 
and,  as  such,  possessed  known  limitations  that  were  described  to  the 
system  users  during  training.  The  Command  reported  that  the  test’s  major 
Umitations  included  (1)  software  that  did  not  contain  the  full  functional 
capability  planned  for  the  initial  operational  test  and  evaluation;  (2)  a  need 
to  reboot  the  system  after  crashes  caused  by  the  use  of  the  computer’s 
alternate  function  key;  (3)  two  changes  in  software  versions  during 
training;  and  (4)  the  fact  that  65  percent  of  the  system  manager  functions 
had  not  been  implemented  or  trained.  Table  1  provides  more  detail  on  the 
customer  test  results. 
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Table  1:  Customer  Test  Operational 
Issues  and  Associated  Army  Test  and 
Experimentation  Command  Comments 


Operational  issue 

Army  Test  and  Experimentation  Command  comments 

Capacity  of  the  system  to 
store  and  process  the 
required  types  and  amounts 
of  data,  including  the  ability 
of  the  staff  users  to  update 
the  required  database 
frequently. 

‘The  system  consistently  locked  up  and  had  to  be 
rebooted  while  the  staff  user  was  attempting  to  process 
.  .  .  data.  This  resulted  in  the  loss  of  all  data  in  working 
files  and  any  data  in  the  queues  awaiting  distribution  or 
processing  to  a  database.” 

“Staff  users  rated  the  systems  capability  to  process  and 
provide  .  .  .  data,  and  to  assist  the  staff  in  the 
performance  of  their  duties  as  marginal." 

"Storing  and  processing  .  .  .  data  was  adequately 
demonstrated  by  [MCS]  for  only  two  functions:  editing 
specified  reports  and  processing  specified  messages. 

.  .  .  application  software  for  the  other  functions  .  .  . 
performed  inconsistently  and  rendered  the  system 
unreliable." 

The  capabilities  of  the  MCS 
network  to  process  and 
distribute  current  and 
accurate  data  using  the 
existing  communications 
systems. 

“The  [system  to  distribute  data]  and  the  distributed 
computing  environment  did  not  work  as  required.  The 
systems  locked  up  and  the  message  handler  backed  up 
(sometimes  with  thousands  of  messages).  The  test  officer 
noted  .  .  .  that ...  the  dedicated  server,  had  a  message 
queue  backlog  of  19,000  messages.  This  situation, 
combined  with  the  necessity  to  reboot  the  system  .  .  . 
throughout  the  test,  caused  backlogged  messages  to  be 
lost.  The  staff  users  were  often  unable  to  initiate  or 
complete  tasks  and  transmit  data  between  the  nodes." 

Impact  of  computer  server 
outages  on  continuity  of 
operations. 

The  Army  Test  and  Experimentation  Command’s  report 
indicates  that  the  third  operational  issue  was  met,  stating 
that  the  success  rate  for  continuity  of  operations  was  100 
percent. 

System  administration  and 
control  capabilities  to 
initialize  the  system,  become 
fully  operational,  and  sustain 
operations. 

“Sixty  five  percent  of  the  system  manager  functions  are 
not  yet  implemented,  and  were  not  trained." 

“The  results  indicate  that  system  administration  and 
control  capabilities  functions  are  incomplete  for  this  build 
of  software.  Additionally,  poor  system  performance,  and 
an  immature  training  program  hampered  the  user’s  ability 
to  sustain  operations." 

Source:  Maneuver  Control  System/Phoenix:  Customer  Test  Report,  Command,  Control,  and 
Communications  Test  Directorate:  Army  Test  and  Experimentation  Command,  1996-CT-1302, 
June  1996. 


In  addition  to  these  findings,  the  mcs  test  officer  stated  the  following: 

“System  performance  degraded  over  time  causing  message  backlogs,  loss 
of  data,  and  numerous  system  reboots.  Over  a  period  of  12  operational 
hours  the  [data  distributing  system]  slowed  down  and  created  message 
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Army  Requested  Flexibility  for 
Operational  Testing  of 
Follow-on  Software 


backlogs  of  up  to  4  hours.  To  remain  functional,  the  entire  network  of 
[mcs]  systems  must  be  shut  down  and  reinitialized  in  proper  sequence.” 

•  “The  staff  users  had  great  difficulty  using ...  [multiple]  applications.” 

•  “The  software  pertaining  to  system  management  functions  was  immature, 
incomplete  and  lacked  documentation.  This  capability  is  critical  to  the 
effective  use  and  operation  of  the  [mcs]  system.” 

Even  though  the  customer  test  did  not  involve  pass/fail  criteria,  based  on 
our  review  of  the  test  report  and  the  test  officer’s  comments,  we  believe 
that  only  the  third  operational  issue — impact  of  computer  server  outages 
on  continuity  of  operations — was  met.  Despite  the  results  of  the  customer 
test  and  the  program’s  prior  history,  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  approved  the  Army’s  plan  to  award  a 
concurrent  contract  for  mcs  Block  IV  software  development — mcs  versions 
12.1,  12.2  and  12.3. 

In  September  1996,  the  Army  awarded  a  contract  for  the  development  of 
MCS  software  versions  12.1,  12.2,  and  12.3  to  a  different  contractor  than  the 
developers  of  mcs  version  12.01.  At  that  time,  version  12.01  was  still 
scheduled  to  undergo  its  initial  operational  testing  in  November  1996.  The 
start  of  the  follow-on  development  could  have  been  timed  to  occur  after 
version  12.01  had  completed  that  operational  testing.  At  most,  this  action 
would  have  delayed  the  contract  award  2  months,  assuming  that  the  initial 
operational  test  had  occurred  in  November  1996  as  scheduled.  However, 
the  contract  was  awarded  before  the  initial  operational  test,  and  the 
planned  5  month  concurrency  in  the  development  of  versions  12.01  and 
12.1  became  18  months  when  the  operational  test  slipped  to  March  1998. 

The  current  program  schedule  indicates  that  (1)  version  12.1  is  expected 
to  undergo  its  operational  assessment/test  about  1  year  after  the  fielding  of 
version  12.01  is  started  and  (2)  version  12.1  fielding  is  to  be  done  5  months 
after  initial  operational  capability  of  version  12.01  is  achieved.  If  the 
scheduled  version  12.01  operational  test  and  evaluation  shps  again  and  the 
version  12.1  contractor  is  able  to  maintain  its  development  schedule, 
version  12.1  could  become  available  before  version  12.01. 

By  May  1997,  the  Army  requested  dod  approval  of  a  revised  acquisition 
program  baseline  that  changes  the  planned  follow-on  operational  test  and 
evaluation  of  versions  12.1, 12.2,  and  12.3  to  operationad 
assessments/operational  tests.  Program  officials  said  that,  although  the 
name  of  the  tests  had  changed,  the  planned  scope  of  the  tests  had  not. 
However,  the  officials  said  that  the  name  change  complies  with  guidance 
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from  DOT&E,  which  lists  multiple  levels  of  operational  test  and  evaluation 
(from  an  abbreviated  assessment  to  full  operational  test)  and  outlines  a 
risk  assessment  methodology  to  be  used  to  determine  the  level  of  testing 
to  be  performed.  The  officials  further  stated  that  the  use  of  the  generic 
term  operational  test/operational  assessment  permits  possible  changes  to 
the  level  of  testing  for  version  12.1  and  follow-on  software  increments 
based  on  the  risk  assessment  process. 

The  contractors  competing  for  the  mcs  Block  IV  (mcs  versions  12.1, 12.2, 
and  12.3)  development  were  given  access  to  the  government’s  12.01  code 
and  allowed  to  reuse  as  much  of  it  as  they  chose.  The  Block  IV  developer 
is  not  required  to  reuse  any  of  version  12.01.  Rather,  the  Block  IV  contract 
requires  the  development  of  software  to  provide  specific  functions.  Given 
that  (1)  version  12.01  software  has  not  passed  or  even  undergone  an  initial 
operational  test  and  evaluation  and  (2)  the  mcs  Block  IV  contractor 
building  version  12.1  is  not  the  contractor  that  is  building  version  12.01 
and  is  only  required  to  develop  the  version  12.1  to  provide  specified 
functions,  we  believe  that  the  version  12.1  development  effort  should  not 
be  viewed  as  building  upon  a  proven  baseline.  Instead,  it  should  be  viewed 
as  a  new  effort. 

Continuation  of  Current 

Development  Strategy  Could  Be 

Costly 


The  Army’s  current  development  plan  for  version  12.1  and  beyond,  as 
shown  in  figure  1,  continues  an  approach  of  building  a  follow-on  version  of 
software  on  an  incomplete  and  unstable  baseline — ^the  uncompleted 
preceding  version  of  software. 


Page  9 


GAO/NSIAD-98-15  Battlefield  Automation 


B-276830 


Figure  1:  Future  MCS  Software  Development  Schedule 
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Fielding 


Source:  Army. 


Additionally,  according  to  an  official  in  the  dod’s  Office  of  the  Director  of 
Test,  Systems  Engineering,  and  Evaluation,  the  Army’s  development 
process  allows  requirements  that  are  planned  for  one  software  version, 
which  cannot  be  accomplished  in  that  version’s  development  as  planned, 
to  he  deferred  to  a  later  version’s  development.  As  a  result,  this  process 
makes  judging  program  risk  and  total  cost  very  difficult. 

The  MCS  program  has  previously  demonstrated  the  problem  of  deferring 
requirements.  For  example,  during  mcs  version  1 1  development,  we 
reported  that  the  Army  had  deferred  seven  mcs  functions  that  were  to  have 
been  developed  by  June  1992  and  included  in  the  software  version  to 
undergo  operational  testing.®  Even  though  the  version  11  operational  test 
had  slipped  twice,  from  May  1992  to  September  1992  and  then  to 


"Battlefield  Automation:  Planned  Production  Decision  for  Army  Control  System  is  Premature 
(GAO/NSIAD-92-151,  August  10,  1992). 


Page  10 


GAO/NSIAD-98-15  Battlefield  Automation 


B-276830 


May  1993,  the  Army  continued  to  defer  those  fimctions,  and  the 
operational  test  was  planned  for  less  than  the  complete  software  package 
originally  scheduled  to  be  tested. 

In  commenting  on  a  draft  of  this  report,  dod  said  that  they  had  made 
progress  not  reflected  in  that  draft.  Specifically,  they  noted  that  there  were 
no  priority  one  or  two,  and  only  22  priority  three  softwaire  deficiencies 
open  as  of  September  11,  1997,  as  compared  Avith  10  priority  one,  47 
priority  two,  and  67  priority  three  deficiencies  open  on  August  16, 1996. 
While  we  agree  these  results  indicate  that  some  known  problems  have 
been  fixed,  they  provide  no  indication  of  the  number  or  severity  of  still 
unknown  problems.  For  example,  mcs  version  12.01  development  showed 
enough  progress  entering  the  November  1996  scheduled  initial  operational 
test  and  evaluation  to  reach  a  commitment  of  resources  and  personnel. 
However,  that  test  was  later  downgraded  to  a  limited  user  test  because  of 
software  immaturity.  Successful  completion  of  an  initial  operational  test 
and  evaluation  should  provide  a  more  definitive  indication  of  the  mcs 
program’s  progress. 


Buying  Training  Base 
Computers  Before 
Operational  Testing  Is 
Questionable 


Before  the  slip  of  the  mcs  initial  operational  test  and  evaluation  from 
November  1996  to  March  1998,  the  Army  planned  to  acquire  288 
computers — 150  in  fiscal  year  1997  and  138  in  fiscal  year  1998 — for  the  mcs 
training  base.  These  computers  were  to  be  acquired  after  a  full-rate 
production  decision  at  a  total  cost  of  about  $34.8  million — $19.1  million  in 
fiscal  year  1997  and  $15.7  million  in  fiscal  year  1998. 


After  the  initial  operational  test  and  evaluation  slipped,  dod  approved  the 
Army’s  acquisition  of  a  low-rate  initial  production  of  81  computers  in  fiscal 
year  1997  for  a  training  base  operational  assessment.  The  purpose  of  the 
assessment,  which  was  performed  from  February  to  May  1997,  was  to 
judge  the  merits  of  allowing  the  Army  to  procure  the  remaining  computers 
prior  to  successful  completion  of  the  slipped  operational  test.  On  the  basis 
of  the  results  of  that  assessment,  the  Acting  Under  Secretary  of  Defense 
for  Acquisition  and  Technology  authorized  the  Army  in  July  1997  to 
proceed  with  its  acquisition  plans.  The  Acting  Under  Secretary  noted  that 
the  DOT&E  had  reviewed  the  assessment  and  agreed  that  version  12.01  was 
adequate  for  use  in  the  training  base. 


The  Acting  Under  Secretary  also  authorized  the  Army  to  move  the  training 
base  computer  funds  from  the  mcs  budget  to  the  Army’s  automated  data 
processing  equipment  program  budget  hne.  'This  action  was  necessary 
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because,  according  to  both  Army  and  dod  officials,  it  was  determined  that 
the  computers  to  be  acquired  do  not  meet  the  legislated  reasons  in  10 
U.S.C.  2400  for  low-rate  initial  production.  That  legislation  allows  the  early 
acquisition  of  systems  to  (1)  estabUsh  an  initial  production  base, 

(2)  permit  an  orderly  increase  in  the  production  rate  for  the  system  that  is 
sufficient  to  lead  to  full-rate  production  upon  successful  completion  of 
operational  test  and  evaluation,  and  (3)  provide  production-representative 
items  for  operational  test  and  evaluation.  Even  though  the  Army  now 
plans  to  acquire  the  computers  under  a  different  budget  line,  the  intended 
use  of  the  computers  remains  unchanged. 

MCS  program  officials  said  that  the  computers  are  needed  in  the  mcs 
training  base  before  operational  testing  to  adequately  support  future 
fielding  of  mcs  and  the  larger  Army  Battle  Command  System,  of  which  the 
Army  Tactical  Command  and  Control  System  and  mcs  are  key 
components.  This  rationale  is  the  same  one  the  Acting  Under  Secretary 
cited  in  his  July  1997  memorandum.  In  that  memorandum,  he  stated  that 
the  “requirement  to  train  Army-wide  on  commercial  equipment  is  a 
recognized  requirement  not  only  for  mcs  but  for  a  host  of  other  digital . . . 
systems.”  The  Acting  Under  Secretary  further  noted  that  the  funds  to  be 
moved  were  for  equipment  needed  to  support  integrated  training  of 
multiple  systems  throughout  the  Army  and  concluded  that  “training  on  a 
digital  system,  even  if  it  is  not  the  system  that  is  ultimately  fielded,  is 
important  to  the  Army  in  order  to  assist  in  making  the  cultural  change 
from  current  maneuver  control  practice  to  a  digitized  approach.” 

mcs  program  officials  stated  that  the  mcs  course  curriculum  needs  to  be 
developed  and  that  equipping  the  training  base  before  the  completion  of 
operational  testing  avoids  a  2-year  lag  between  the  completion  of 
operational  testing  and  the  graduation  of  trained  students.  The  officials 
also  commented  that  the  computers  could  be  used  elsewhere,  since  they 
would  be  compatible  with  other  Army  programs. 

The  legislated  requirement®  that  major  systems,  such  as  mcs,  undergo 
initial  operational  test  and  evaluation  before  full-rate  production  serves  to 
limit  or  avoid  premature  acquisitions.  The  Army  has  had  previous 
experience  acquiring  ineffective  mcs  equipment,  which  is  indicative  of  the 
need  for  adequate  testing  before  systems  are  fielded.  In  July  1990,  the 
Army  began  withdrawing  over  $100  million  of  militarized  mcs  hardware 
from  the  field  due  to  both  hardware  and  software  deficiencies. 
Additionally,  the  Army  subsequently  decided  not  to  deploy  other  mcs 


'Title  10  U.S.C.  2399. 
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equipment  it  had  procured  for  light  divisions  at  a  cost  of  about  $29  million 
because  the  equipment  was  too  bulky  and  heavy. 


The  Mcs  program’s  troubled  development  and  acquisition  history  has 
continued  since  the  program’s  1993  reorganization.  However,  the  Army 
awarded  a  new  contract  to  develop  future  software  versions  and  plans  to 
procure  computers  without  fully  resolving  the  problems  of  earlier 
versions.  This  strategy  does  not  minimize  the  possibility  of  future 
development  problems  and  ensure  that  the  Army  will  ultimately  field  a 
capable  system.  Also,  since  mcs  software  version  12.1  is  being  developed 
concurrently  by  a  different  contractor  to  functional  specifications,  it 
would  be  prudent  to  subject  the  version  12.1  software  to  the  level  of 
operational  testing  required  to  support  a  fuU-rate  production  decision,  as 
planned  for  version  12.01.  Accordingly,  we  believe  a  more  appropriate 
strategy  would  require  that  future  software  versions  be  developed  using 
only  fully  tested  baselines,  and  that  each  version  be  judged  against  specific 
pre-established  criteria. 

We  recommend  that  you  direct  the  Secretary  of  the  Army  to 

•  set  specific  required  capabilities  for  each  software  version  beyond  version 
12.01,  test  those  versions  against  specific  pass/fail  criteria  for  those 
capabilities,  and  only  award  further  development  contracts  once  problems 
highlighted  in  that  testing  are  resolved; 

•  perform  a  full  operational  test  and  evaluation  of  mcs  software  version  12.1 
to  ensure  that  it  provides  the  full  capabilities  of  version  12.01;  and 

•  procure  additional  mcs  computers  only  after  an  initial  operational  test  and 
evaluation  and  a  full-rate  production  decision  have  been  completed. 


Conclusions  and 
Recommendations 


Agency  Comments 
and  Our  Evaluation 


In  commenting  on  a  draft  of  this  report,  dod  agreed  wdth  our 
recommendation  that  specific  required  capabilities  for  each  mcs  software 
version  beyond  version  12.01  are  needed,  that  those  versions  should  be 
tested  against  specific  pass/fail  criteria  for  those  capabilities,  and  that  the 
Army  should  not  award  further  development  contracts  until  problems 
highlighted  in  prior  tests  are  resolved,  dod  noted  that  the  Army  has  already 
set  specific  required  capabilities  for  those  software  versions  and  will  test 
those  versions  against  specific  pass/fail  criteria  to  ensure  system  maturity 
and  determine  that  the  system  remains  operationally  effective  and 
suitable,  dod  further  stated  that  it  will  not  support  the  award  of  further 
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development  contracts  until  the  Army  has  successfully  resolved  any 
problems  identified  during  the  testing  of  related,  preceding  versions. 

DOD  partially  agreed  with  our  recommendation  that  the  Army  be  directed 
to  perform  a  full-operational  test  and  evaluation  of  MCS  software  version 
12.1  to  ensure  that  it  provides  the  full  capabilities  of  version  12.01.  dod 
stated  that  the  Army  will  comply  with  dod  regulation  5000.2R  and  will 
follow  guidance  from  Director  of  Operational  Test  and  Evaluation,  which 
lists  multiple  levels  of  operational  test  and  evaluation  (from  an 
abbreviated  assessment  to  full  operational  test)  and  outlines  a  risk 
assessment  methodology  to  be  used  to  determine  the  level  of  testing  to  be 
performed,  dod  did  not,  however,  indicate  whether  it  would  require  the 
Army  to  conduct  a  full  operational  test.  We  continue  to  believe  that  the 
version  12.1  development  effort  should  not  be  viewed  as  building  upon  a 
proven  baseline.  Instead,  version  12.1  development  should  be  viewed  as  a 
new  effort.  As  a  result,  we  still  believe  that  the  prudent  action  is  to  require 
that  version  12.1  be  subjected  to  the  same  level  of  operational  test  and 
evaluation  as  version  12.01,  the  level  required  to  support  a  full-rate 
production  decision. 

DOD  agreed  with  our  recommendation  that  it  direct  the  Army  to  not 
procure  more  MCS  computers  until  the  completion  of  an  initial  operational 
test  and  evaluation  and  a  full-rate  production  decision.  It  stated,  however, 
that  no  further  direction  to  the  Army  is  needed  as  it  had  already  provided 
direction  to  the  Army  on  this  issue.  Specifically,  the  Department  stated 
that  it  has  directed  the  Army  to  extract  the  training  base  computers  from 
the  MCS  program  and  to  not  procure  or  field  more  MCS  hardware  to 
operational  units  until  successfully  completing  an  initial  operational  test 
and  evaluation.  Our  recommendation,  however,  is  not  limited  to  the 
hardware  for  operational  units,  but  also  encompasses  the  computers  the 
Army  plans  to  buy  for  the  training  base.  Given  the  program’s  prior  history 
and  the  fact  that  the  training  base  computers  are  not  needed  to  satisfy  any 
of  the  legislated  reasons  for  low-rate  initial  production,  we  continue  to 
believe  that  the  Army  should  not  be  allowed  to  buy  those  computers  until 
MCS  has  successfully  completed  its  initial  operational  test  and 
evaluation — the  original  plan  prior  to  the  MCS  initial  operational  test  and 
evaluation’s  multiple  schedule  slips. 

dod’s  comments  are  reprinted  in  their  entirety  in  appendix  I,  along  with 
our  evaluation.  In  addition  to  those  comments,  we  have  revised  our  report 
where  appropriate  to  reflect  the  technical  changes  that  dod  provided  in  a 
separate  letter. 
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Scope  and 
Methodology 


To  determine  whether  the  current  Mcs  software  development  strategy  is 
appropriate  to  overcome  prior  problems  and  to  determine  whether  the 
Army  should  procure  207  new  computers  for  the  expansion  of  the  mcs 
tradning  base,  we  interviewed  responsible  officials  and  analyzed  pertinent 
documents  in  the  following  dod  offices,  all  in  Washington,  D.C.:  Director 
of  Operational  Test  and  Evaluation;  Director  of  Test,  Systems  Engineering, 
and  Evaluation;  Assistant  Secretary  of  Defense  for  Command,  Control, 
Communications,  and  Intelligence;  Under  Secretary  of  Defense 
(Comptroller);  and  Defense  Procurement.  In  addition,  we  interviewed 
responsible  officials  and  analyzed  test  reports  from  the  office  of  the 
Army’s  Project  Manager,  Operations  Tactical  Data  Systems,  Fort 
Monmouth,  New  Jersey;  and  the  Army’s  Operational  Test  and  Evaluation 
Command,  Alexandria,  Virginia.  To  meet  our  second  objective,  we  also 
interviewed  responsible  officials  and  analyzed  pertinent  documents  from 
the  Army’s  Combined  Arms  Center,  Fort  Leavenworth,  Kansas. 

We  conducted  our  review  from  March  to  September  1997  in  accordance 
with  generally  accepted  government  auditing  standards. 


We  are  sending  copies  of  this  report  to  the  Chairman  and  Ranking 
Minority  Members,  Senate  and  House  Committees  on  Appropriations, 
Senate  Committee  on  Armed  Services,  and  House  Committee  on  National 
Security;  the  Director,  Office  of  Management  and  Budget;  and  the 
Secretary  of  the  Army.  We  will  also  make  copies  available  to  others  on 
request. 

As  you  know,  the  head  of  a  federal  agency  is  required  by  31  U.S.C.  720  to 
submit  a  written  statement  on  actions  taken  our  recommendations  to  the 
Senate  Committee  on  Governmental  Affairs  and  the  House  Committee  on 
Government  Reform  and  Oversight  not  later  than  60  days  after  the  date  of 
this  report.  A  written  statement  must  also  be  submitted  to  the  Senate  and 
House  Committees  on  Appropriations  with  the  agency’s  first  request  for 
appropriations  made  more  than  60  days  after  the  date  of  the  report. 
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Please  contact  me  at  (202)  512-4841  if  you  or  your  staff  have  any  questions 
concerning  this  report.  Major  contributors  to  this  report  were 
Charles  F.  Rey,  Bruce  H.  Thomas,  and  Gregory  K.  Harmon. 

Sincerely  Yours, 


Allen  Li 

Associate  Director 
Defense  Acquisitions  Issues 
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Comments  From  the  Department  of  Defense 


Note:  GAO  comments 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  comments  1  and  2. 


ACQUISITION  AND 
TECHNOLOGY 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  30301-3000 


October  2,  1997 


Mr.  Allen  Li 

Associate  Director,  Defense  Acquisition  Issues 
National  Security  and  International  Affairs  Division 
U.S.  General  Accounting  Office 
Washington,  DC  20548 

Dear  Mr.  Li: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  General  Accounting  Office 
(GAO)  draft  report,  “BATTLEFIELD  AUTOMATION;  Software  Problems  Hinder 
Development  of  Army’s  Maneuver  Control  System,”  dated  August  29,  1997  (GAO  Code 
707242/OSD  Case  1453). 

The  DoD  generally  concurs  with  all  GAO  recommendations  with  comments  (Enclosure 
1).  Testing  will  be  conducted  in  accordance  with  DoD  5000.2-R  to  ensure  system  maturity,  and 
to  determine  operational  effectiveness  and  suitability  prior  to  fielding.  The  DoD  has  directed  the 
Army  not  to  procure  or  field  more  MCS  hardware  to  operational  units  until  they  have 
successfully  completed  an  Initial  Operational  Test  and  Evaluation  (lOT&E)  and  Milestone  III 
decision. 

The  draft  GAO  Report  does  not  acknowledge  the  improvements  made  to  MCS  version 
12.01  since  the  Limited  User  Test  (LUT)  in  October  and  November  1996,  and  should  be  updated 
to  reflect  the  current  status  of  MCS  software.  These  and  additional  technical  comments  on  the 
draft  report  have  been  provided  under  separate  cover. 

Sincerely, 


/  -LJ  >  j 

Patricia  Sanders 
Director,  Test,  Systems 
Engineering  and  Evaluation 


Enclosure: 
As  stated 
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Appendix  I 

Comments  From  the  Department  of  Defense 


Now  on  p,  13. 


Now  on  p.  13. 


See  comment  1 . 


GAO  DRAFT  REPORT  DATED  AUGUST  29, 1997 
(GAO  CODE  707242)  (OSD  CASE  1453) 

“BATTLEFIELD  AUTOMATION:  SOFTWARE  PROBLEMS  HINDER 

DEVELOPMENT  OF  ARMY’S  MANEUVER  CONTROL  SYSTEM” 

DEPARTMENT  OF  DEFENSE  COMMENTS 

*  >4<  ♦ 

RECOMMENDATIONS 

o  RECOMMENDATION  1:  The  GAO  recommended  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Army  to  set  specific  required  capabilities  for  each 
software  version  beyond  version  12.01,  test  those  versions  against  specific  pass/fail 
criteria  for  those  capabilities,  and  not  award  further  development  contracts  until  problems 
highlighted  in  that  testing  are  resolved,  (p.  16/GAO  Draft  Report) 

DoD  RESPONSE.  Concur.  The  Army  has  set  specific  required  capabilities  for 
software  versions  beyond  version  12.01  in  the  OSD  approved  MCS  Block  IV  software 
development  contract.  This  contract  calls  for  delivery  of  three  successive  MCS  versions 
(Version  12.1 ,  12.2  and  the  objective  Version  12.3)  over  a  five  year  period.  Each  version 
meets  a  specific  set  of  Block  IV  requirements.  The  Army  will  test  these  versions  against 
specific  pass/fail  exit  criteria  to  ensure  system  maturity  and  determine  that  the  system 
remains  operationally  effective  and  suitable.  The  DoD  will  not  support  award  of  further 
development  contracts  until  the  Army  has  successfully  resolved  any  problems  identified 
in  Block  IV  testing.  Consequently,  there  is  no  requirement  for  additional  DoD  direction 
to  the  Army. 

o  RECOMMENDATION  2:  The  GAO  recommended  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Army  to  perform  a  full  operational  test  and  evaluation 
of  Maneuver  Control  System  (MCS)  software  version  12.1  to  ensure  that  it  provides  the 
full  capabilities  of  version  12.01.  (p.  16/GAO  Draft  Report) 

DoD  RESPONSE.  Partially  Concur.  For  both  developmental  test  and 
evaluation  (DT&E)  and  operational  test  and  evaluation  (OT&E),  the  Army  will  comply 
with  the  process  outlined  in  Section  3.4  of  DoD  5000.2-R,  dated  March  15,  1996.  To 
determine  the  appropriate  level  of  operational  testing  for  each  software  version,  the  Army 
will  follow  guidance  provided  in  the  Director  Operational  Test  and  Evaluation  (DOTE) 
memorandum.  Subject:  “Guidelines  for  Conducting  Operational  Test  and  Evaluation  for 
Software  Intensive  System  Increments,”  dated  October  10,  1996.  The  MCS  Block  IV 
software  development  contract  will  build  on  the  capabilities  of  version  12.01  and  will 
achieve  planned  functionality  (in  three  version  deliveries)  identified  in  the  MCS 
Operational  Requirements  Document. 
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0  RECOMMENDATION  3:  The  GAO  recommended  that  the  Secretary  of 
Defense  direct  the  Secretary  of  the  Army  not  to  procure  more  MCS  computers  until  an 
initial  operational  test  and  evaluation  and  a  full  rate  production  decision  have  been 
Now  on  p.  13.  completed,  (p.  16/GAO  Draft  Report) 

DoD  RESPONSE.  Concur.  The  OSD  Acquisition  Decision  Memorandum, 
dated  July  16,1 997,  directed  the  Army  to  extract  the  training  base  content  from  the  MCS 
program.  It  also  directed  that  they  not  procure  or  field  more  MCS  hardware  to 
operational  units  until  successfully  completing  an  lOT&E  and  Milestone  III  decision.  As 
See  comment  2.  a  result,  the  Army  revised  the  MCS  Acquisition  Strategy  to  reflect  this  change. 

Therefore,  there  is  no  further  requirement  for  additional  DoD  direction  to  the  Army. 
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GAO  Comments 


The  following  are  gao’s  comments  on  the  Department  of  Defense’s  (dod) 
letter  dated  October  2, 1997. 


1.  In  partially  agreeing  with  this  recommendation,  dod  states  that  the  Army 
will  comply  with  dod  regulation  5000.2R  and  will  follow  guidance  from 
Director  of  Operational  Test  and  Evaluation — guidance  which  lists 
multiple  levels  of  operational  test  and  evaluation  (from  an  abbreviated 
assessment  to  full  operational  test)  and  outlines  a  risk  assessment 
methodology  to  be  used  to  determine  the  level  of  testing  to  be  performed. 
dod  does  not,  however,  indicate  how  they  agree  or  disagree  with  our 
recommendation  or  state  whether  they  wUl  implement  the 
recommendation.  As  we  stated  in  the  body  of  this  report,  given  that  a 
different  contractor  is  building  version  12.1  under  a  requirement  to 
provide  specific  functionality,  we  believe  that  this  development  effort 
should  not  be  viewed  as  budding  upon  a  proven  baseline.  Instead,  version 
12.1  development  should  be  considered  a  new  effort.  As  a  result,  we 
continue  to  believe  that  it  is  prudent  to  require  that  version  12.1  be 
subjected  to  the  level  of  operational  test  and  evaluation  required  to 
support  a  full-rate  production  decision. 

2.  dod’s  direction  to  the  Army  only  partially  implements  our 
recommendation.  Our  recommendation  is  not  Umited  to  the  hardware  for 
operational  units,  but  also  encompasses  the  computers  the  Army  plans  to 
buy  for  the  training  base.  We  continue  to  believe  that  the  Army  should  not 
be  allowed  to  buy  the  planned  training  base  computers  until  Mcs  has 
successfully  completed  its  initial  operational  test  and  evaluation — ^the 
original  plan  prior  to  the  mcs  initial  operational  test  and  evaluation’s 
schedule  slips.  The  training  base  computers  are  not  required  to  satisfy  any 
of  the  three  purposes  the  law  indicates  for  low-rate  initial  production — ^to 
(1)  establish  an  initial  production  base,  (2)  permit  an  orderly  increase  in 
the  production  rate  for  the  system  sufficient  to  lead  to  full-rate  production 
upon  successful  completion  of  operational  test  and  evaluation,  and 

(3)  provide  production-representative  items  for  operational  test  and 
evaluation.  Since  the  training  base  computers  are  not  needed  to  satisfy 
one  of  the  above  legislated  conditions,  we  continue  to  believe  that  the 
Army  should  refrain  from  buying  any  additional  mcs  computers  prior  to  a 
full-rate  production  decision. 
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